Statistical Named Entity Recognizer Adaptation

Named entity recognition (NER) is a subtask of widely-recognized utility of information extraction (IE). NER has been explored in depth to provide rapid characterization of newswire data (Sundheim, 1995; Palmer and Day, 1997). The NER task involves both identification of spans of text referring to named entities, and categorization of these entities into classes based on the role they fill in context. The sentence “Washington announced that Washington ate seven hotdogs in Washington” provides an example in which a single name can arguably refer to three different entities: an organization, a person, and a location. Following the paradigm introduced by Ramshaw and Marcus (1999), many researchers reduce the NER problem to a word-tagging problem, and address it with techniques similar to those used for part of speech tagging (Meteer et al., 1991; Brill, 1995). Borthwick explores the maximum entropy approach in his dissertation (1999). Collins and Singer (1999) investigate semi-unsupervised methods for named entity categorization. Cucerzan and Yarowsky (1999) produce a unified technique for producing NER systems for several languages, utilizing extensive bootstrapping from small amounts of supervised data with an EMstyle algorithm. Miller et al. (2000) produce a statistical Hidden Markov Model (HMM) for NER which is similar to the one used by Palmer et al. (1999); the latter system, named phrag, is the NER engine utilized in the work described in this paper. The experiments described herein explore unsupervised approaches to NER, with an eye toward using unannotated corpora consisting of a few hundred million words. Recent word sense disambiguation results suggest that some simple techniques can scale well with increased data sizes (Banko and Brill, 2001). This paper presents several experiments in adapting a HMM-based named entity recognizer to a target data set. Our core learning engine is a wordbased HMM, and we show two techniques, informed smoothing and iterative adaptation, for incorporating unsupervised data into the model, which provide overall gains in performance.